BUG: SparseSeries.shift may raise NameError or TypeError #12908

sinhrks · 2016-04-16T12:11:50Z

no existing issue
tests added / passed
passes git diff upstream/master | flake8 --diff
whatsnew entry

Fixed following bugs on current master. Also, moved TestSparseArrayIndex to test_libsparse

1. shift

pd.SparseSeries([1, 2, 3], fill_value=0).shift(1)
# NameError: global name 'kwds' is not defined

pd.SparseSeries([1, 2, 3]).shift(1)
# TypeError: %d format: a number is required, not float

2. dtype

from pandas.sparse.array import IntIndex
arr = pd.SparseArray([1, 2], sparse_index=IntIndex(4, [1, 2]), dtype=None)
arr.dtype
# dtype('int64')

arr.values
# array([-9223372036854775808,                    1,                    2,
#        -9223372036854775808])

# Expected outputs
arr.dtype
# dtype('float64')

arr.values
# array([ nan,   1.,   2.,  nan])

jreback · 2016-04-17T13:57:35Z

pandas/sparse/tests/test_array.py

@@ -159,6 +22,58 @@ def setUp(self):
        self.arr = SparseArray(self.arr_data)
        self.zarr = SparseArray([0, 0, 1, 2, 3, 0, 4, 5, 0, 6], fill_value=0)

+    def test_constructor_dtype(self):
+        arr = SparseArray([np.nan, 1, 2, np.nan])
+        self.assertEqual(arr.dtype, np.float64)


I would test the fill_value on all of the these (e.g. nan as the default, 0 on next example)

Added more tests in test_shift_nan. Is this what you mean?

no I meant that you are asserting the fill_value in some of the tests here (as well as the dtype), but might want to assert fill_value for all (an example will be when we have nan fill values that we DO coerce on creation), e.g. say we have sparse datetimes (we don't really support this ATM)

pd.SparseArray([pd.NaT, pd.Timestamp('20130101'), pd.Timestamp('20130102'), pd.NaT], fill_value=np.nan)

@jreback Thanks. ATM we can't create non-numeric SparseArray (#11856). Your example raises TypeError.

no I get it. I just want a line that says self.assertTrue(np.isnan(arr.fill_value)) on the tests that don't have this.

I see. Added missing checks.

jreback · 2016-04-20T01:07:53Z

thanks!

sparse is really looking nice!

sinhrks added Bug Indexing Related to indexing on series/frames, not to indexes themselves Dtype Conversions Unexpected or buggy dtype conversions Sparse Sparse Data Type labels Apr 16, 2016

sinhrks added this to the 0.18.1 milestone Apr 16, 2016

jreback reviewed Apr 17, 2016
View reviewed changes

sinhrks force-pushed the sparse_shift branch from 4e1aedd to c2c7923 Compare April 17, 2016 14:19

BUG: SparseSeries.shift may raise NameError or TypeError

5a0adfa

sinhrks force-pushed the sparse_shift branch from c2c7923 to 5a0adfa Compare April 18, 2016 20:48

jreback closed this in 5d8a935 Apr 20, 2016

sinhrks deleted the sparse_shift branch April 20, 2016 01:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BUG: SparseSeries.shift may raise NameError or TypeError #12908

BUG: SparseSeries.shift may raise NameError or TypeError #12908

sinhrks commented Apr 16, 2016

jreback Apr 17, 2016

sinhrks Apr 17, 2016

jreback Apr 17, 2016 •

edited

Loading

sinhrks Apr 17, 2016 •

edited

Loading

jreback Apr 18, 2016

sinhrks Apr 19, 2016

jreback commented Apr 20, 2016

BUG: SparseSeries.shift may raise NameError or TypeError #12908

BUG: SparseSeries.shift may raise NameError or TypeError #12908

Conversation

sinhrks commented Apr 16, 2016

1. shift

2. dtype

jreback Apr 17, 2016

Choose a reason for hiding this comment

sinhrks Apr 17, 2016

Choose a reason for hiding this comment

jreback Apr 17, 2016 • edited Loading

Choose a reason for hiding this comment

sinhrks Apr 17, 2016 • edited Loading

Choose a reason for hiding this comment

jreback Apr 18, 2016

Choose a reason for hiding this comment

sinhrks Apr 19, 2016

Choose a reason for hiding this comment

jreback commented Apr 20, 2016

jreback Apr 17, 2016 •

edited

Loading

sinhrks Apr 17, 2016 •

edited

Loading